Bandit Algorithms in Game Tree Search: Application to Computer Renju∗

Multi-armed bandit problem is to maximize a cumulated reward by playing arms sequentially without prior knowledge. Algorithms for this problem such as UCT have been successfully extended to computer GO programs and proved significantly effective by defeating professional players. The goal of the project is to implement a Renju AI based on Monte Carlo planning that is able to defeat the oldest known alpha-beta pruning AI. In this project we will give an implementation of the UCT algorithm and attempt to improve the minimax tree search by modeling the dependencies between arms. Our work is greatly inspired by the theoretical advances in GPTS(Gaussian Process for Tree Search) where the mean reward function is drawn from a Gaussian Process. Applying GPTS to game tree search is by no means trivial and it would be interesting to see how it works in practice.